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Abstract — Just as the Hamming weight spectrum of a linear 
block code sheds light on the performance of a maximum 
likelihood decoder, the pseudo-weight spectrum provides insight 
into the performance of a linear programming decoder. Using 
properties of polyhedral cones, we find the pseudo-weight spec- 
trum of some short codes. We also present two general lower 
bounds on the minimum pseudo-weight. The first bound is based 
on the column weight of the parity-check matrix. The second 
bound is computed by solving an optimization problem. In some 
cases, this bound is more tractable to compute than previously 
known bounds and thus can be applied to longer codes. 

I. Introduction 

Inspired by the success of iterative message-passing de- 
coding, there have been numerous efforts to understand its 
behavior [11], [4], [5]. Recently, Koetter and Vontobel [6] 
presented an analysis of iterative decoding based on graph- 
covering. This analysis explains why the notion of pseudo- 
codeword arises so naturally in iterative decoding. They also 
showed that the set of pseudo-codewords can be described as 
a polytope, which they called the fundamental polytope. 

At the same time, Feldman [3] introduced a decoding 
algorithm based on linear programming (LP). This decoder 
was successfully applied to low-density parity-check (LDPC) 
codes and many turbo-like codes. It turns out that this decoding 
method is closely related to the analysis by Koetter and 
Vontobel. In particular, the feasible region of Feldman's linear 
program agrees with the fundamental polytope. 

For a given channel, a pseudo-weight can be defined for 
each pseudo-codeword. The pseudo-weight spectrum relates 
to the performance of an LP decoder in very much the 
same way as the Hamming weight spectrum does to the 
performance of a maximum likelihood decoder. Thus it is 
of interest to find the pseudo-weight spectrum of a code. 
For very short codes, this might be achieved by employing 
a technique related to the dual polyhedral cone given in [2]. 
Some examples of pseudo-weight spectrum calculation for the 
additive white Gaussian noise channel will be demonstrated 
in Section IV. For longer codes, computing the entire pseudo- 
weight spectrum becomes intractable and we have to judge 
the performance of a code from bounds on the minimum 
pseudo-weight. Some techniques to compute lower bounds on 
minimum pseudo-weight were presented in [10]. In Section V, 
we will discuss two new lower bounds. One is based on 



the column weights of the parity-check matrix. The other is 
computed by solving an optimization problem. 

II. Linear Programming Decoding 

Let C be a binary linear code of length n. Such a code is 
a linear subspace of F£. In this paper, we will also view C 
as a subset of R™. Suppose that a codeword y is transmitted 
through a binary-input memoryless channel and r is the output 
of the channel. The log-likelihood ratio 7 is defined as 



Any codeword x € C that minimizes the cost -f T x is a 
maximum-likelihood (ML) codeword [3]. Thus ML decoding 
is equivalent to solving the problem: 



minimize ^ T x 
subject to x 6 C. 



Letting H be a parity-check matrix of C, the feasible set of this 
problem can be relaxed to a polytope P = {x <G l n | Bx < 
b, < Xi < 1}, where the matrix B and the vector b are 
determined from H as follows [3]. For a row h of H, let 
U(h) be the support of h, i.e., the set of positions of 1 in h. 
Then Bx < b consists of the following inequalities: for each 
row h of H and for each set V C U(h) such that \V\ odd, 

i€V i£U(h)\V 

Now, the problem is transformed to a linear program. This 
approach, introduced by Feldman [3], is called linear program- 
ming (LP) decoding. The polytope P is called the fundamental 
polytope by Koetter and Vontobel [6] (Fig. 1). It has the 
property that a 0-1 vector is in the polytope if and only if 
it is a codeword of C. Thus, if a 0-1 vector is a solution to 
the linear program, it must be an ML codeword. However, 
unlike in ML decoding, the output of the LP decoder may not 
be a 0-1 vector, in which case the decoder simply declares an 
error. 

III. Error Analysis 

The fundamental polytope has a symmetry property that 
allows us to assume without loss of generality that the all-zeros 



codeword is transmitted, provided that the channel is a binary- 
input output-symmetric channel [3]. Roughly speaking, the 
fundamental polytope "looks" the same from every codeword. 
Therefore we assume that the all-zeros codeword is transmitted 
and remove from the linear program all inequality constraints 
that are not active at the origin. (An inequality f(x) < a is 
active at a point x* if f(x*) = a.) We obtain a new linear 
program, which we will call LPCONE: 



minimize 7 T x 

subject to x £ K = {x £ W l \ Ax < 0, x t > 0}, 



where A is the submatrix of B corresponding to zero-entries 
of b. The feasible set K is a polyhedral cone and it is called 
the fundamental cone [6] (Fig. 1). 
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Fig. 1 . Conceptual pictures of the fundamental polytope and the fundamental 
cone 

Assuming that the zero codeword is transmitted, the prob- 
ability of error of LPCONE is the same as that of the linear 
program in the previous section. To see this, suppose that the 
zero vector is a solution to the original linear program. Then 
7 T x > for all x £ P. It can be shown that a vector x is 
in K if and only if ax £ P for some a > 0. It follows that 
the zero vector is also a solution to LPCONE. The converse 
is immediate since P C K. Hence, it is sufficient to consider 
LPCONE to evaluate the performance of the LP decoder. For 
this reason, we will mainly consider LPCONE instead of the 
original linear program. 

To compute the probability of error, we need to find the set 
K* such that the zero vector is a solution to the linear program 
if and only if 7 £ K* . To describe the set K*, we proceed as 
follows. Let W = {w\, . . . , w m } be the set of "generators" 
of the cone K (Fig. 2), i.e., a vector a; is in if if and only if 
x can be written as a nonnegative linear combination of the 
generators: x = aiWi + ■ ■ ■ + a m w m , where a-y, . . , , a m are 
some nonnegative real numbers. The zero vector is a solution 
to the linear program if and only if -f T x > for all x £ K. 
It can be shown that this condition is equivalent to 7 1 x > 
for all x £ W. Hence, the decoding is successful if and only 
if the log-likelihood ratio 7 is in 

K* = {z £ R™ I z T x > for all x £ W} (2) 

(Fig. 2). The set K * is called the dual cone of K [1]. 

IV. Pseudo-Weight on the AWGN Channel 

For ML decoding over a memoryless channel, the probabil- 
ity of error of a code is largely determined by its Hamming 
weight spectrum. For iterative and LP decoding, it has been 




z T Wi = 

Fig. 2. The generators of the cone K and the dual cone K* 

observed that the notion of "pseudo-weight" is more appropri- 
ate for determining the probability of error [11], [4], [6]. The 
definition of pseudo-weight varies with the channel; we will 
study only the pseudo-weight on the additive white Gaussian 
noise (AWGN) channel. 

Before stating the definition of pseudo-weight, we will first 
extend the discussion in the previous section for the AWGN 
channel. We hope that by doing so the intuition behind the 
definition of pseudo-weight will be more apparent. 

Consider the discrete-time AWGN channel in Fig. 3. Each 
bit of the codeword y is modulated to +1 and —1 and then 
corrupted by additive white Gaussian noise with variance a 2 . 
The received vector is denoted by r. It can be shown that the 
log-likelihood ratio 7 is given by 7$ = (2/cr 2 )r 2 ;. We recall 
from the previous section that LP decoding is successful if 
and only if 7 £ K*, the dual of the fundamental cone. Since 
scaling the cost function by a positive scalar does not change 
the solution to a linear program, the decoding is successful if 
and only if r £ K*. 
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Fig. 3. Binary-input AWGN channel 

We recall that W = {wi, . . . , w m } is the set of generators 
for the fundamental cone. The dual cone K* can be described 
by the hyperplanes z T Wi = as in (2). The transmitted 
vector corresponding to the all-zeros codeword is the all-ones 
vector, which will be denoted by 1. The Euclidean distance 
from the all-ones vector to the hyperplane i is l T w;j/|| Wi\\, 
where || • || denotes the Euclidean norm (Fig. 4). If the noise 
perturbs the transmitted vector by this distance in the direction 
perpendicular to the hyperplane, the LP decoder will fail. 

By way of comparison, consider ML decoding. In ML 
decoding, the Voronoi region for the all-zeros codeword is 
defined by the hyperplanes separating the all-ones vector and 
the transmitted vectors of the other codewords. Let y be a 
codeword with Hamming weight d. The distance from the 
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Fig. 4. Distances from the all-ones vector 1 to the hyperplanes defining the 
dual cone K* 



all-ones vector 1 to the hyperplane separating 1 and the 
transmitted vector corresponding to y is \/d. This relationship 
between the Euclidean distance to the hyperplane and the 
Hamming weight of y motivates the definition of pseudo- 
weight of a vector x for the AWGN channel [11], [4], [6]: 
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p(x) 
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Ml 

If £ is a 0-1 vector, its pseudo-weight is equal to its Hamming 
weight. 

Given a generator of K, computing its pseudo-weight is 
trivial. However, given a cone, finding its generators is a very 
complex task. A straightforward way is to add an equality 
constraint to LPCONE, such as l T x = 1, so that the feasible 
set becomes bounded. Then change the problem into the 
standard form and find all "basic feasible solutions," which 
correspond to the corner points of the feasible region. This 
involves (^) ways of choosing "basic variables," where N 
and M are the number of variables and the number of equality 
constraints of the modified linear program. (For more details, 
refer to any linear optimization book, e.g., [7].) 

A simpler way to find the set of generators of a cone is 
presented in [2]. However, the complexity is still very high 
and thus the algorithm can only be applied to very short 
codes. Using this algorithm, we computed the histograms of 
the pseudo-weights of the (7, 4) and (15, 11) Hamming codes, 
shown in Fig. 5. 

V. Bounds on Minimum Pseudo-Weight 

Tanner [8] gave several lower bounds on the minimum 
Hamming weight of linear block codes. One of them, called 
the bit-oriented bound, is a function of the column and row 
weights of the parity-check matrix H and the eigenvalues of 
H T H. Another one, called the optimization distance bound, 
is computed by solving an optimization problem. Two lower 
bounds on minimum pseudo-weight were presented in [10]. 
One of them is similar to the bit-oriented bound of Tanner [8]. 
The other, called the LP-based bound, is computed by solving 
a linear program derived from the fundamental cone. 

In this section, we will present two lower bounds on 
minimum pseudo-weight. Before doing so, we prove two 
propositions which are useful in establishing the bounds. 
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Fig. 5. Pseudo-weight spectra of the (7, 4) and the (15, 11) Hamming codes 



Proposition 1: Let K be a polyhedral cone and W its set 
of generators. Then 

min p(x) = minp(x) = min p(x). 

xdW xdK x£K, l T x=l 

The proof of Proposition 1 is given in the appendix. Since 
p(x) = ((l T a;)/||a;||) 2 , it follows from Proposition 1 that the 
problem of minimizing the pseudo-weight over W becomes 
the following non-convex problem, which we call MaxNorm: 







maximize 


INI 2 


subject to 


x G K , 




l T x = 1. 



Our lower bounds are obtained by relaxing this difficult 
problem to an easier one, particularly the one in Proposition 2 
below. Since the feasible set of MaxNorm is contained in 



the feasible set of the relaxed problem, 



,'112 



, where 



x* and x' are the solutions to MaxNorm and the relaxed 
problem, respectively. Therefore the minimum pseudo-weight, 
which equals l/||a;*|| 2 , is lower bounded by 1 / 1 1 a; 7 1 1 2 . 

Proposition 2: Let a.i, 1 < i < n, be nonnegative real 
numbers. Consider the optimization problem: 



maximize 


H 2 


subject to 


x G E n 




< Xi < cti for all 1 < i < n, 




l T x - 1. 



Suppose that 2~27=i a i — 1 an d a i are ordered such that ct\ > 
a2 > ... > Q/ n . Let j be the first index such that ai + 

a\ — ■ ■ ■ — aj-i 



3 > 1. Then the maximum of the objective function is 

2 , (1 .,1 



a 



Proof: Let x satisfy l T x = 1 and < Xi < a; for all 
1 < i < n. Let k be the smallest index such that x k < ak- 
Let m be the largest index such that x m > 0. We define a 
new vector x' as follows. 

Case 1: a k - x k < x m . Let x' k = a k and x' m = x m - a k + 

Xk- 

Case 2: — .t^, > x m . Let xj, = + x m and = 0. 

For the other indices t ^ k,t ^ m, let x' t = x t . It can be 
shown that l T x' = 1 and < x\ < a,. Moreover, we claim 
that ||a/|| > ||x||. We repeat this assignment until x' = x. 
(The algorithm terminates since either k is increased or m is 
decreased in the next iteration.) The final vector x* satisfies 
x* = a v for 1 < v < j — 1, Xj = 1 — a± — ■ ■ ■ — otj-x, and 
x* = for j + 1 < v < n, and the proposition follows. 

To prove the claim, consider the two cases. 

Case 1: Qife — Xfe < x m . 

IM| 2 -N| 2 = {x' k f + {x' m f-xl-xl 

'2,1 , \2 2 2 

= a k + (x m - a k + x k ) - x k - x m 
= 2(a k - x m )(a k - Xk) > 0. 

Case 2: — x k > x m . 

\\x'\\ 2 - \\x\\ 2 = (x k + x m f - x 2 k - x 2 m > 0. 



A. Bound from Column Weight 

If the Tanner graph of a parity-check matrix H has no cycle 
of length four, it is well known that the minimum Hamming 
distance is lower bounded by the minimum column weight of 
H plus one. This is true for the minimum pseudo-weight as 
well. 

Theorem 3: Suppose that any two columns of the parity- 
check matrix H have at most one 1 in the same position. 
Then the minimum pseudo-weight is lower bounded by the 
minimum column weight plus one. 

Proof: Let m* be the minimum column weight of H. 
The basic idea of the proof is to relax MaxNorm to the 
problem in Proposition 2 where cti = \/(m* + 1). Then the 
theorem will follow. 

Consider the i-th column of H, which is denoted by q. Let 
m be its column weight. Let qy,...,q m be the positions of 1 
in Ci. Since any two columns of H have at most one 1 in the 
same position, the support of the qj-th row can be written as 
Rj U {i}, where Rj n R k = for 1 < k < m, k ^ j. 

Let x £ K and l T x = 1. We recall that K is defined 
by the inequalities (1) that are active at the origin. (These 
are the inequalities with \V\ = 1.) Therefore x satisfies Xi — 
J2k£R x k ^ f° r a U 1 < j < ™- Since the sets Rj are 
pairwise disjoint, 

n ra m 

1 = ^2 xi > Xi + Xk > Xj + Xj = (m + 

1=1 j=l k£Rj j=l 

Hence xi < 1/(to* + 1) for all 1 < i < n. Then the theorem 
follows from Proposition 2. ■ 



B. Relaxation Bounds 

Suppose that we choose a set S 3 K and relax MaxNorm 
to the following: 



maximize 


IMI 2 


subject to 


x e S, 




1 T X = 1. 



The set S should be as small as possible; however, the new 
problem should be easy to solve. A good choice for S is the 
hyper-rectangle 

S = {xe E" | < Xi < oti}, 

where on is the maximum of the objective function of the 
following linear program: 



maximize 


x t 


subject to 


xeK, 




l T x = 1. 



Without loss of generality, assume that a\ > a.2 > ■ ■ ■ > a n . 
Let j be the first index such that ai + ■ ■ ■ + otj > 1. From 
Proposition 2, 

min pseudo-weight > — —, 

We call this the first-order bound. 
A more complex choice for S is 

S = {x e K" I < Xi < ai, x l + Xj < /3 hj }, 

where ai are computed in the same way as above, and /3ij 
are computed by replacing the objective function of the above 
linear program by Xi + Xj. Unfortunately, the problem is 
hard to solve for this set S. We approach the problem by 
partitioning the feasible region into n sub-regions based on 
the maximum entry of x. We obtain n sub-problems; the sub- 
problem k, 1 < k < ?i, is the following: 



maximize 


IMI 2 
















subject to 


< Xi 


< ai 


for 


all 


1 


< i 


< 


n. 




X | X ' 


< 


for 


all 


1 


< i 


< 


n, 








1 < 


3 


< 


n, 








l T x = 


1, 














Xk > x 




for 


all 


1 


< i 


< 


n. 





















Then we relax each sub-problem by omitting the inequalities 
Xi + xj < Pij whenever i ^ k and j ^ k. 



maximize 


IMI 2 








subject to 


< Xi < ai 


for all 1 < i 


< 


71. 




Xk +Xi< /3 k ,i 


for all 1 < i 


< 


n, 




l T x = 1, 










x k > x t 


for all 1 < i 


< 


it. 



If we fix Xk, then the relaxed sub-problem has the same form 
as when S is a rectangle, which we can solve by Proposi- 
tion 2. Thus we can compute the maximum of the relaxed 
sub-problem for a fixed Xk- By varying Xk and computing 
the corresponding maximum, we can solve the relaxed sub- 
problem k. Taking the maximum of ||:r|| 2 over all relaxed sub- 



problems, we obtain an upper bound of 



for the original 



problem, which will lead to a lower bound for the minimum 
pseudo-weight. We call this the second-order bound. 
We have a few remarks regarding these lower bounds: 

• Any lower bound for minimum pseudo-weight is also a 
lower bound for minimum Hamming weight. 

• To compute the first-order and the second-order bounds, 
we need to solve as many as n and n(n+ 1)/2 linear pro- 
grams respectively. Each of these linear programs has n 
variables. This is in contrast to the optimization distance 
bound and the LP-based bound discussed earlier, which 
are computed by solving one linear program whose worst- 
case number of variables is quadratic in the codeword 
length. 

• The number of linear programs to be solved for the first- 
order and the second-order bounds can be reduced if the 
code has some structure. 

Next we compute these bounds for a class of group- 
structured LDPC codes presented in [9]. A particular sequence 
of codes in this class has constant column weight 3 and 
constant row weight 5, with rate approximately 2/5. The 
bounds on minimum pseudo-weights for these codes are 
shown in Table I. The upper bound is computed by using 
the MATLAB optimization toolbox to find a good feasible 
solution to MaxNorm. Note that for the code with length 
155, Vontobel and Koetter [10] have found tighter lower and 
upper bounds: 10.8 and 16.4 respectively. 

TABLE I 

Lower and upper bounds on minimum pseudo-weights of 
Tanner's LDPC codes with rate approximately 2/5 



length 


first-order 


second-order 


upper bound 


155 


8.3 


9.7 


17.0 


305 


11.5 


13.8 


20.1 


755 


13.0 


14.0 


27.6 


905 


17.6 


21.5 


39.4 



Appendix 
Proof of Proposition 1 

In this appendix we present a proof of Proposition 1 . First, 
we need the following lemma. 

Lemma 4: Let a, b, c, d be positive real numbers. Then 



b + d 



> min 



Proof: Suppose that a/b < c/d. Then 

d 



a 
- c 



a 

a + c 



> 



> 



> 



b + d 

The case c/d < a/b is similar. ■ 
Proof of Proposition 1: First, we will show 
that mirizgvy p(x) = mm m £Kp{x), Since W C K, 
mhi xe w p{x) > mm X £K p(x). Conversely, let x,y,z € K 
and a, /3 > with x = ay + j3z. Then l T x = al T y + (31 T z. 

It follows that 



By the triangle inequality, 



p(x) 



> 



\ X \\<a\\y\\+f3\\z\ 
al T y + f31 T z s 9 




Titt ' liTir r = min Mi/).p( z )}. 



where the latter inequality follows from Lemma 4. Since every 
x E K can be written as a nonnegative linear combination 
of the generators, there is a generator w <G W such that 
p{x) > p(w). Therefore m.m X £w p(x) < min x g_K- p{x). Fi- 
nally, mm xe K p(x) = min^g^ \T x=1 p(x) since the pseudo- 
weight is invariant under scaling, i.e., p(ax) = p(x). ■ 
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